Information Theoretic Divergences With Application To Information Fusion

Many modern day collaborative systems / system of systems (SoS) rely heavily on the sharing of information in order to improve performance, manage resources, and maximize overall capability. These types of systems are characterized by many decentralized nodes which can either all be identical or are divided into a finite set of specialized types. When information is shared within any SoS, the overall performance hinges on its ability to correctly associate the information received. The primary hypotheses evaluated by any SoS after the receipt of new information are (i) Does this information belong to an entity previously observed; or (ii) Does this information belong to a new entity, which includes both real and false entities. In order to evaluate these hypotheses, an association discriminator needs to be defined and evaluated which properly assesses the received information. This paper defines the properties required of a data association discriminator, highlights the measures of information which satisfy these properties, and develops the corresponding gating equations for use during the data association process. The architecture upon which a SoS is developed and designed plays a fundamental role when selecting an appropriate association discriminator. Some of these architectural considerations are (a) The type of SoS, decentralized or centralized; (b) The interface to and the performance of the individual data sources; (c) The capacity for managing the data sources and the receipt of information; (d) The computational resources available; and (e) The desired level of robustness to data association errors. This paper discusses various measures of information commonly used between and within the components of these types of systems and compares and contrasts their behavior in a common framework with a focus on the data association problem. Commonly used measures of information, namely, differential entropy, mutual information, and divergences, along with the logarithmic score function are examined analytically, the advantages and disadvantages of each are discussed, new results are derived and presented, and several simulation examples are presented which illuminate the use of these measures of information. Lastly, contrary to the current literature, it is demonstrated that the Kullback-Leibler discriminator / divergence is not an optimal data association discriminator.